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Abstract 

We study the convergence of the false discovery proportion (FDP) of the Benjamini-Hochberg 
procedure in the Gaussian equi-correlated model, when the correlation p m converges to zero as 
the hypothesis number m grows to infinity. By contrast with the standard convergence rate m 1 ^ 2 
holding under independence, this study shows that the FDP converges to the false discovery rate 
(FDR) at rate {min(m, l/p m )}'^ 2 in this equi-correlated model. 
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1. Introduction 

When testing simultaneousl y a large numb er m of null hypotheses, a popular global type I 
error, that can be traced back to Seeger ( 1968), is the false discovery proportion (FDP), defined 
as the ratio of the number of erroneous rejections to the number of rejecti ons. The average of this 
random variable, called the false discovery rate (FDR, introduced by Benjamini and Hochberg 



( 1995)), h as been studied by a considerable number of works, see for instance 



Sarkar (2008) and 



Blanchard and Roquainl ([2009) for a review. However, studying the FDR is not sufficient to catch 



the full behavior of the FDP, for instance a FDR control does not prevent the FDP from having 
large variations. Therefor e, some other studies aim to direct l y control the upper-quantile of the 
FDP distribution, see e.g. IGenovese and Wassermanl (120061) : iLehmann and Romano (2005), or 
to directly compute the d istribution of the F DP, either non-asymptotica l lv IChi and Tanl (120080: 
Roquain a nd Villersl ( 120 101) . or asvmptoticallv lGenovese and Wassermanl (120041) . Recently jNeuvial 
(2008, 2009) computed the asymptotic distribution of the FDP actually achieved by the Benjamini- 
Hochberg (BH) procedure (and some other adaptive procedures) under independence of the p- 
values. It is prov ed that the FDP converges to the FDR at the parametric rate -\/m. Furthermore, 
Farcomenil J2007I) showed that this convergence is unchanged under a specific short-range de- 



pendency between the /j-values. 

In this paper, we are interested in studying the convergence of the FDP of the BH procedure 
in the model where the test statistics have exchangeable Gaussian errors, with equi-correlation 
p (allowing for instance l ong-range depen d encies ) . This mode l has become quite standard in 
multiple testing (see e.g. IBeniamini et alJ (120061) : iFinner et al.l (120071) ). as it is a very simple 
instance of dependent /rvalue model. From an intuitive point of view, the test statistics can be 
seen as independent test statistics plus a disturbance variable whose importance depends on the 
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value of p. Whenp e (0, 1) is fixe d with m (and in the "ideal" setting where the p-values under 



the alternative are all equal to zero), Finn er et al.1 (|2007) proved that the FDP of the BH procedure 



converges to a non-deterministic random variable that still depends on the disturbance variable. 
When p,„ — > 0, we show here that this disturbance variable has no effect on the limit of the FDP 
anymore, which equals n^a (where jiq is the proportion of true nulls), but can still have an effect 
on the asymptotic variance of the FDP or even on the convergence rate. More precisely, when 
p,„ — > our main result states that {min(m, l/p ; „)} 1 ^ 2 (FDP„, - n^a) -^ N(0, V) holds for a given 
V > 0; in comparison with the independent case, we may distinguish the two following cases, 
recovering all the possible convergence regimes of p,„ to zero: 

• when lim m mp m — e [— 1, +oo), the limit of the FDP and the convergence rate are the 
same as in the independent case. The asymptotic variance V is larger if 9 > 0, smaller if 
6 < and is the same whenever = (i.e. mp m —> 0). 

• when lim,„ mp m = +oo and lim m p m = the convergence rate is p m ' instead of m 1 ^ 2 . 

On the one hand, this shows that the FDP of the BH procedure is still well concentrated around 
n^a under weak equi-correlation such that p,„ = (9(1 /m). On the other hand, this puts forward 
that the concentration of the FDP of the BH procedure around the FDR may be arbitrarily slow 
when p,„ — > 0, which is a striking result that has not been reported before to the best of our 
knowledge. Therefore, our recommendation is that the BH procedure can be used under Gaussian 
equi-correlation when p m = 0{ 1 jni) (including the case of a negative equi-correlation) but should 
be used carefully as soon as mp m — > oo, as the actual convergence rate of the FDP to the FDR 
might be much slower. 

The paper is organized as follows: Section|2]presents the model, the notation and the main 
result. The latter is proved in Section[3] including a generalization to any "regul ar" thr esholding 
procedure, recovering the so-called 7To-adaptive procedures studied in iNeuviall d2008l) . Finally, 
some further points in connexion with our methodology are discussed in Section[4] 

2. Setting and main result 

We observe Xj = t,- + T,-, 1 < i < m, where the parameter of interest is (t,),- e {0, p}" 1 (for a 
given p > 0) and the (unobservable) error vector {Y\, ..., Y,„) is an exchangeable Gaussian vec- 
tor with EFi = Oand VarFj = 1. We let p,„ = Cov(Ti,F 2 ) e [-(m- l)" 1 , 1]. We consider 
the problem of the one-sided testing of the null "r, = 0" against the alternative "r, = p", si- 
multaneously for any 1 < i < m. To test each null, we define the p-value p, = <£>(Xj), where 
O(z) = P(Z > z) is the standard Gaussian upper-tail function. The c.d.f. of each p-value is 
denoted by Go(f) = t under the null and by G\(t) = 0(0 _1 (f) - p) under the alternative. The 
number of true nulls is denoted by mo(m) = \{i \ r, = 0}| and is assumed to be of the form 
[mnoi for a given proportion of true null rq e (0, 1) independent of m. The "mixture" c.d.f. 
of the p-values is denoted by G(f) = ^oGo(f) + (1 - no)G\(t). Next, we define the e.c.d.f.'s 
@0,m(0 = (moim))- 1 2£i l{Tj = 0}l{ Pi < t), Gi^f) = (m - mo(m))- 1 £*, 1{t; > 0}l{p ; < t) and 
G B (0 = m- 1 £»! l{ Pi < t). 



Given a pre-specified level a e (0, 1), the procedure of lBeniamini and H ochberg ( 1995J) can 
be defined as the procedure rejecting the nulls corresponding to p, < T BH (G m ) where the (data- 
driven) threshold 7~ B "(G,„) is maxjf e [0, 1] | G m (f) > t/a). Next, the false discovery proportion 



at a given threshold t e [0, 1] is defined as the proportion of true nulls among the hypotheses 
having a p-value smaller than or equal to t: 



FDP m (?) = 



{1 < i<m | T, =0,pi < t}\ m (m) &o,m(0 



{1 < i <m \pi < t}\ V 1 



G m (0 v mr 



where | • | denotes the cardinality function. 
We now state our main result. 

Theorem 2.1. There is a unique point t* e (0, 1) such that G(t*) — t* /a and we have 

(i) if \im m mp m - 6 e [— 1, +oo), then 

^(FDP m (T BH (G m )) - noa) ^ N 1 0, m^ 2 ^- + 0^f— 2 

\ t* 2n(t*y 

(ii) if lim m mp m — +oo and lim m p m — 0, then 



2 2 
n a „-(0-'(f*)) 2 



p-V\FDP m (T BH (& m )) - n Q a) - N 0, 



_2„.2 



*\2' 



-(0-'(/*)) 2 



27T(f*) 



(1) 



(2) 



3. Proof of Theorem [27T1 



3.1. A more general result 

In what follows, we denote the space of functions from [0, 1] to R which are right-continuous 
and with left-hand limits (Skorokhod's space) by D(Q, 1) and the space of continuous functions 
fro m [0, 11 to R by C(0, 1). The method for proving our result relies on the methodology let down 
by iNeuviall ( 2008 ) which consider the case of a general threshold fu nction T : Z)(0, 1) — > [0, 1] 
which is Hadamard differentiable at G, tangentially to C(0, 1) (see Ivan der Vaartl (11998b for a 
formal definition). The proof of Theorem 12.11 is presented here as a consequence of a more 
general theorem, true for any such threshold. The derivative of the threshold T at G, which is 
a continuous linear form on C(0, 1), is denoted by Tq. According to the Riesz representation 
theorem, the continuous linear form Tq can be written as 7"c(F) = J. F{t)Tc(dt), where we 
identified the linear form Tc and the corresponding signed measure. 

Theorem 3.1. Let T : D(0, 1) — > [0, 1] be Hadamard differentiable at G, tangentially to C(0, 1), 
with derivative T/j. Let q(t) = not/G(t) for t > 0, let t* — T(G) and assume t* > 0. We set 

<To = *EQ?0^ + q(t+)n f G , £ = - *%SP fr + #**)(! " ^)t G and 



c(T) = (2. 



my 1 ' 2 f e-i 
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,-^(^») ; 



&(*) + (2 
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,-k(<lr l <f)-tf 



Cxidt); 



'f ' 

J[0,1] 2 



cr 2 (r)=^o 1 f (5Vf-^o(^Xo(fl f O + (l-^or 1 I (Gi(jV0-Gi(j)GiW)fi(rfj)fi(*). 

J[0,1] 2 '"'"' 

77zen the following holds: 

(i) if\\m m mp m - 6 e [— 1, +oo), 



y/nl(FDP m (T(G m )) - q(t*)) -^ N (0, tr 2 (r) + #c(r) 2 ) ; (3) 
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(ii) if lim m mp m — +00 and lim m p m — 0, 

Pm " 2 (FDP m (T(G m )) - q(t*)) ~» N (0, c(7~) 2 ) . (4) 



Let us now check that Theorem l2. ll follows from Theorem l3.ll Froir uNeuviall (12008b Corol- 



lary 7.12, T BH : F h-> maxjf e [0, 1] | F(t) > t/a} is Hadamard differentiable at G, tangentially to 
C(0, 1), with derivative T<5 H = (l/a - G(t*))- l 8,*. Moreover, t* = max{f e [0, 1] | G(t) > t/a} is 
positive, because lim,^o+ t/G(t) - 0. Also, since G(t*) - t* /aandq(t*) - (l/a , -G(t*))noa 2 /t*, 
we may check that £1 = and £0 = (noa/t*)5 t * in the above theorem, which leads to Theorem l2.ll 

5.2. Proof of Theorem \3.1\ 



Let us now prove Theorem 13. II First write FDP m (7"(G,„)) = (mo(m)/TO)7r lv F(Go, m ,Gi ?m ), 
where for any Fq,F\ in D(0, 1) with F(T(F)) > (letting F = ;roFo + (1 _ ^o)F\), we put 
*P(Fo,Fi) = 7Tq pfj-Qr!) • From standard computations, *P is Hadamard differentiable at (Go,Gi), 
tangentially to C(0, l) 2 and the derivative takes the form, for (H , Hi) e C(0, l) 2 , *P CojCl (Ho, Hi) = 
«(f*)(l - 9 (r*))(» - |P) + q(t*)t G (H), where // = iro#o + (1 -;r )#i. Applying the func- 
tional Delta m ethod, this leads to the following useful result, which was essentially stated in 
Neuviall J2008I) . 



Proposition 3.2. Lef T : D(0, 1) — > [0, 1] foe Hadamard differentiable at G, tangentially to 
C(0, 1), with derivative 7"q. Let q(t) — itot/G(f) for t > 0, let t* — T(G) and assume t* > 0. If 
for a given sequence a m — > 00 with a m — o(m), 



Go, m - Go 
Gi, m -G\ ) 



S /W «- (5 ) 

1 ' 



where the convergence in distribution is relative to the Skorokhod topology and where Wo and 
Wi are processes with continuous paths, then we have 

a m (FDP m (T(G m )) - q(t*)) ~> X, (6) 

where X — ^b(Wo) + fi(Wi) and go, C,\ are defined as in Theorem \3.1\ 

A convergence of the type (0 in the particular Gaussian equi-correlated model is stated in 
Lemma 1331 Using Proposition [372] this proves that (|6]l holds both in the cases (i) and (ii) with 
a m - ^m and a m - p m , which respectively leads to ([3J and <(4j (the variance computations are 
straightforward). 

3.3. Convergence of the e.c.d.f's in the Gaussian equi-correlated model 

Lemma 3.3. Let (Zo, Zj , Z) be a random variable such that Zo = n^ B, Z\ = (1 — 7To)~ 1/7 B o 
G\, B being a standard Brownian bridge on [0, 1], Zo is independent from Zi, Z ~ N(0, 1), 
Cov(Z,Z W) = (2n)- 1/2 exp(-{(I>- l (t)} 2 /2) andCow(Z,Zi(t)) = (2n)- 1/2 exp(-[<I>- 1 (f)-fi} 2 /2). 
Let also U ~ N(0, 1) be independent of the vector (Zo,Zi,Z). Then we have the following 
convergences in law for the Skorokhod topology: 

(i) iflim m mp„ 

,> ) ; (7) 



= Oe [-l,+oo), 






1— I Go,,,) - Go 

ym\ j-j 


-w 


Z 4- (Z - Vl + 6/17) 6 0)-' 

v Zi + (z - Vi + eu) <i> (d)- 1 - 
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(ii) if lim m mp m — +00 and lim m p m — 0, 



-1/2 I ^0,m - Go 



(/•Jo^-' -//) 



-1 .a I- (8) 



To prove Lemma [331 first remark that the distribution of the X,-'s may be realized as Xj - 
VT^(£ -|) + yl(\+(m-\)p m )lm U+uMTi > 0), where (ft, ...,&,, £/) are all i.i.d. Af(0, 1) 
variables and £ denotes the empirical mean of them's. LetG (f) = (mo(m))" 1 2i:r>=o 1{^(£) - f K 
G' u „(f) = (m - mo(m))- 1 2, :T , >0 1{<B(£ + p) < f} and 

/ m a,C/,p m ) = (l-p m )- 1/2 (<5- 1 (0- V(l + (m-l)p m )/mt/). 

The process (Go, m - Go, Gi, m - Gi) is then equal to V,„ + W m where 



V m (t) 



(g; ,„ - G0(O(/ m (f, f/,p m ) + f -p(i - p,„)- 1/2 + /i» 



(&' 0m -G )(®(f m (t L U,p m ) + ?)) 
i(f m (t,U, Pm )+^-M(l- 

W(t )J <t>(f m (t,u, Pm )+ib-t 

m{ \ nxfjf, u, Pm ) + £ - Mi - p m ) _1/2 ) - OKO-HO - m) 

Next, applying Donsker's theorem, we derive -\/m(G' 0m - Go,G' lm - Gi,£)~-»(Zo, Zi,Z), 
where (Zo,Zi,Z) is defined as in Lemma [33] Since p,„ — > 0, the inverse functions of t i-» 
<t>(/ m (f, £/,p m ) + 1) and f 1-4 <D(/ ffl (f, C7,p m ) + 1 - p(l - p,„)" 1/2 + //) converge uniformly on [0, 1] 
to the identity a.s. Therefore, applying the Skorokhod's representation theorem, we get 

V^(V m ,?)^(Z ,Zi,Z). (9) 

Let us now consider the case (i), in which lim m mp m -de [— 1, +cx>). In that case, a standard 
reasoning involving Taylor expansions of O and y i-» <t>(_y«l> : (f)) leads to 

with remainder terms satisfying ||/?o,mlloo V ||i?i, m ||oo — > in probability. Since U is independent 
of all the ot her va riables, we derive from © that yfm(V m , W,„) ~> (Z , Zi, (Z - Vl + 0£/)) ® o 
<J> _1 ,(Z - Vl + 01/)) <i> o (0 _1 - //)). This implies ©. Consider now the case (ii), in which 
lim„, mp m - +00 and lim„, p m - 0. In that situation, we deduce from <j9j that p m ' V m converges 

— 1 /2 — 

in probability to 0. Furthermore, using that p m ' £; tends to zero in probability, we obtain that 



with remainder terms satisfying ||T"o,mlloo V ||7"i >m ||oo — > in probability. This implies (|S). 

4. Discussion: FDP convergence in the case p,„ = p 6 (0, 1) 

When p m — p G (0, 1), we c annot expect that the FDP concentrates around the FDR as in 



Theorem l2.1l (see e.g. iFinner et alj (2007) Theorem 2. 1). As a consequence, even if the FDP has 

5 



a mean below n^a (because the false disc overy rate of the BH procedure is below n^a for each 



m for PRDS statistics, see Theorem 1.2 in Beniamini and Yekutieli (2001)), the FDP can exceed 



noa + s (s > 0) with a probability that does not vanish when m grows to infinity. 

We claim here that in the ideal situation where the parameters of the model no, fi, p are 
perfectly known, it is possible to modify the p-values so that the FDP convergence to the FDR 
keeps the parametric convergence rate yfm. For this, we replace each test statistic X t by Xj = 
^m/((m - 1)(1 - p))(Xj — X+ (1 - 710)//), so that (X\, ...,X m ) is a Gaussian vector with variances 
equal to 1, equi-correlationp„, = -(m - l)" 1 and means EX,- = sjm/((m - 1)(1 - p))tj. We build 
the corresponding p-values by letting p, = <b(Xi), which are uniform under the null and have 
the c.d.f. G\_ m (t) - <&(<b~ l (t) -Ji m ) forp m = (m/(m- l)) 1/2 /z(l -p)~ 1/2 under the alternative. 
Although the latter depends (slightly) on m, we easily check that our methodology applies using 
Gi(t) — <&(<b~ x {t) -Ji) for /I = yu(l - pY 1 ^ 2 and that the following convergence holds: 



V»i(FDP„, - TToa) ~-> N , n^a 



1 _ t* ~2fy2 \ 

2_ V _ n U -(O-'ft*)) 7 



where FDP,,, denotes the FDP of the BH threshold T™ used with the p-values p,'s and where 
t* G (0, 1 ) is the unique point t e (0, l)satisfying^ f+(l-^o) < l ) ( ( l )_1 (0-^(l-p) _1/2 ) = ?/ff(which 
depends on p). Of course, while this p-value modification greatly improves the concentration of 
the FDP, this approach is oracle because jiq, /u, p are generally unknown. A correct estimation of 
the model parameters within such a procedure stays an open issue. 
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